feat: per-model control of which sampling params are sent to the backend - #237
Merged
Conversation
…t to the backend (incl. review fixes)
Owner
|
Merged, will be released in v2.0.119 AI-summary Adds per-model control over which sampling parameters are sent to the LLM backend. Auto-config now probes each model to detect params the backend rejects (with a control-probe guard added during review to avoid false attribution), and the builder strips them from outgoing requests. The UI exposes per-param "Send" checkboxes plus a recovery control for
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Some backends reject sampling parameters they don't support (e.g.
temperature,top_p,top_k,reasoning_effort), returning HTTP 400 and blocking the conversation. OpenFox already stripped these via anomitParamsfield set by auto-config, but users had no manual control over which params are sent.This PR adds a "Send" checkbox next to each sampling parameter in the provider model editor (Advanced section), so users can explicitly enable/disable individual params per model. Unchecking a param adds it to
omitParamsand strips it from outgoing requests; re-checking removes it.What changed
SamplingParamFieldcomponent with a "Send" checkbox per param (Temperature, Top P, Top K, Max tokens). Unchecking disables + dims the input and adds the param key toomitParams. Re-checking re-enables it. Pre-existingomitParamsentries (e.g.reasoning_effortfrom auto-config) are preserved on toggle.reasoning_effortwithout re-running auto-config.SamplingParamFieldcomponent. Net LOC reduction.How it works
The
omitParamsfield was already wired through the full stack (ModelConfig → provider-manager → client-pure → request body). This PR exposes it in the UI so users can control it manually, without relying solely on auto-config detection.Related issues
reasoningin message history, handled by the existingsendReasoningInMessagesflag — but the underlying need for per-field control is the same.)AI-Enhanced Development
Cache Impact
omitParamsstrips generation-time parameters (temperature,top_p,top_k,max_tokens,reasoning_effort) from the request body. These are not part of the cached prompt prefix (system prompt + tools + messages) or the tool fingerprint. The dynamic context hash and KV cache are unaffected.Test plan
npm run typecheck— cleannpm run lint— cleannpx prettier --check(modified files) — cleannpx vitest run web/src/components/shared/ProviderModal.test.tsx— 18 passednpx vitest run src/server/llm/client-pure.test.ts src/server/providers/auto-config.test.ts src/server/provider-manager.test.ts— 75 passed